Causal Classification: Treatment Effect vs. Outcome Estimation

The goal of causal classification is to identify (classify) individuals whose outcome would be positively changed by a treatment. Large-scale examples include targeting (online) advertisements and targeting retention incentives to high-value customers or employees who are at high-risk for attrition. Causal classification can be remarkably difficult for multiple reasons, most importantly because data at best show each individual under only one treatment condition, so it is impossible to know for certain which individuals had a positive outcome due to the treatment. Therefore, the potential outcomes for each treatment condition are estimated either explicitly or implicitly in causal classification. Then, these estimates are used to decide which individuals to target with a treatment. Curiously, in practice we see causal classification problems being treated simply as outcome prediction rather than as a causal inference task, e.g., will someone purchase if shown the ad? We might write that off as naivete, but perhaps there is a good reason. In this paper, we undertake a theoretical analysis comparing treatment effect estimation vs. simple outcome prediction when addressing causal classification. The analytical results show a bias/variance trade-off: because treatment effect estimation depends on two outcome estimates instead of one, the larger variance may lead to higher misclassification error than the (biased) outcome prediction approach. As the analytical results include approximations, we next introduce a flexible simulation environment for experimenting with causal classification; simulation results support the analytical results. The bottom line is that using outcome prediction sometimes is indeed preferable to treatment effect estimation, even when the best-possible models are used for both approaches and there are no estimation challenges (such as confounding). Specifically, outcome prediction is preferable when positive outcomes are (1) very rare, (2) difficult to predict, and when (3) treatment effects are small.